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ABSTRACT 

This paper describes the collaborative development of an 
instrument to assess graduates from 1 to 5 years after teacher education 
program completion. The study shows how a consortium of colleges and 
universities used the same instrument to gather data for research and program 
improvement. Examples of research results in the University of New Hampshire 
Program are discussed, using the total data from the consortium as a frame of 
reference. The survey instrument is attached. In 1990 a consortium of 11 
universities and schools across the United States was formed, and consortium 
members used the Survey developed at the University of New Hampshire to 
collect data on their graduates. Data for the first round of research came 
from 1,394 graduates from 6 institutions. The second round collected data 
from approximately 3,050 graduates from 11 institutions. Each institution was 
given data on its own program and on the total of all programs. Data from the 
University of New Hampshire suggested many significant differences in 
graduates of 4-year and 5-year programs, and the second round of study 
continued to look at the differences among 4-year, 5-year, 5-year integrated 
undergraduate-graduate, and post BA entry programs. The ways such data can be 
used are discussed, and some comparisons are made between findings for the 
University of New Hampshire and those for consortium members overall. (SLD) 
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Introduction 

This paper describes the collaborative development of an instrument to assess 
graduates from one to five years after teacher education program completion. The paper 
shows how a consortium of colleges and universities used the same instrument to gather 
data for research and program improvement. Examples of research results in the UNH 
Program will be discussed using the total data from the consortium as a frame of 
reference. A survey instrument is described. 

Collaborative Development of an Instrument to Survey Graduates from UNH 

A group of colleagues in the UNH Teacher Education Program organized in the 
late 1980’s to investigate the UNH teacher education program and its theory-practice 
integration. The group named itself the Teacher Education Research Group (TERG). 
Michael Andrew, Director of Teacher Education at UNH, had begun in 1974 to collect 
annual program assessment data from interns and cooperating teachers at the end of the 
internship year, which is the capstone and endpoint of the five-year program. He also 
collected one-year graduate follow up data on all graduates. Andrew had data on the 
percent of UNH graduates that entered the profession every year - a percentage that was 
far higher than the national average. We wanted to know more about our own graduates’ 
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performance in the classrooms and schools in which they taught. Teacher-as-leader was 
a theme of our five-year program since its inception in 1974. We wanted to know more 
about the ways in which our graduates took on leadership roles in the classrooms and 
schools in which they taught. The UNH Teacher Education Research Group developed 
the prototype for a Graduate Survey. 

A Consortium of Universities Refines the Instrument to Survey All Graduates 

In 1990 a consortium of eleven universities and schools across the United States 
was formed. The Survey was modified and used by these institutions to collect data on 
their graduates of the prior five years, from 1985-1990. Meetings at UNH, University of 
Florida at Gainsville, and AACTE annual conferences enabled members of the 
consortium to meet to conduct analyses and future planning needed during the first Five 
-Year Graduate Follow up Study and a subsequent study by the consortium in 1996 that 
surveyed graduates from 1990-1995. Michael Andrew directed the research effort. 

The initial consortium consisted of Austin College, Sherman TX; Drake 
University, DesMoines, IA; the University of Florida, Gainesville, FL; the University of 
Kansas, Lawrence, KS; the University of New Hampshire, Durham, NH; the University 
of Vermont, Burlington, VT; Texas A & M University, the University of Nebraska, 
Oakland University, Oakland, MI; and the University of Rhode Island, and the University 
of Virginia. The data from the 1 st round of the research project comes from 1394 
graduates from 1985-1990 at these 11 institutions. 

The 2 nd round of the study surveyed graduates from the period 1990-1995, 
approximately 3,050 graduates from 11 institutions. Six (6) institutions from the 1 st 
round of the study [ Austin College, Drake University, the University of Florida; the 
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University of Kansas; the University of New Hampshire; and the University of Vermont] 
were joined by the following 5 institutions, Idaho State University, Pocatello, ID; 
Louisiana State University, Baton Rouge, LA; Truman State University, Kirksville, MI; 
The University of Arkansas, Fayetteville, AS; and the University of Tennessee, 

Knoxville, TN. 

The consortium members in the 1 st round of the study shared a special interest in 
comparing graduates of four-year and extended teacher education programs. Eight (8) of 
the 11 institutions in the 1 st round of the study had integrated five-year teacher education 
programs (programs for preservice teachers which combine professional studies at both 
the undergraduate and graduate levels). Four (4) of the 8 (Austin College, University of 
New Hampshire, University of Kansas, and University of Florida) had pioneered the 
integrated five-year program movement and had been making plans for this joint research 
venture as early as 1984. Several consortium members had fifth-year, post-BA entry 
programs. Other institutions with four-year programs were considering restructuring 
their teacher preparation programs. A preliminary study of graduates of four-year and 
five-year programs at the University of New Hampshire (Andrew, 1990) had suggested 
many significant differences in graduates of four and five year programs. The 2 nd round 
of the study continued to look at differences among four-year, five-year integrated 
undergraduate-graduate, and post-BA entry programs. 

Each institution was given data on its own program and data on the total of all 
programs. In both rounds of the study there was concern about the consequences of 
publicizing comparative data. Comparative data was shared anonymously in papers at 
AACTE and subsequent publications. Some institutions partnered to investigate 



ERIC 



4 



comparative data among themselves on particular issues of interest. Outcomes of the 1 st 
round of the study showed that there is a range of quality among all program types. This 
finding and concerns about use of comparative data led the consortium to choose to 
emphasize best practices. The 2 nd round of the study was named the Benchmark Project, 
and its goal was to identify and share outstanding practices in preservice teacher 
education. 

The Survey of Graduates 

The Survey of Graduates provides research data for each institution and 
comparative data by which each institution can compare itself to all the institutions in the 
consortium. While they differ in size and affiliation, all of the consortium institutions 
have been involved in innovative efforts to restructure and strengthen their teacher 
education programs. The consortium members share an interest in follow-up of graduates 
of teacher education programs to gain evidence of performance of their students and the 
effectiveness of their programs. All the institutions were interested in their graduate’s 
reasons for teaching, graduates’ teaching activities or approaches in the classroom, the 
climate of the schools in which graduates were teaching. Some of the institutions had 
been focused on development of teacher leadership, and they were very interested in the 
items on the survey related to leadership of graduates among their peers. Also included 
were demographic characteristics, information about graduates’ current teaching 
situations, aspects of their teacher education programs, and opinions toward a variety of 
educational issues. 

In the 2 nd round of the study a revised Survey of Graduates (see attachment) gave 
new emphasis to leadership practices of teachers and school context factors that might 
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support teacher leadership (teachers taking the lead in curriculum development, 
instructional improvement of self and others, and in supporting school reform). For 
example, Likert items were added to the leadership practices in Survey question #18 (see 
#18L, #18M, #18N, #180, & #18P) and Survey question #17 (see #17F and #17N). Five 
items were added to the school climate variable (see Items #1 1C, #1 ID, #1 II, #1 1 J, & 
#11K). 

In the 2 nd round of the study items were also added to the Survey of Graduates to 
get feedback on the effectiveness of teacher education programs (for example, #34, #35, 
and #36). 

Comparative Analyses Using the Data from The Graduate Survey of 11 Institutions 

Date from all 1 1 institutions can be analyzed in a number of ways. 

When the survey data from all 1 1 institutions was used to compare five-year 
extended programs with four-year programs, the research findings showed that graduates 
of extended programs of teacher preparation excel over four year program graduates on 
every major variable in the survey; this includes entry into teaching, retention in teaching, 
career satisfaction, satisfaction with teacher preparation programs, as well as graduates’ 
self-rating of classroom performance, professional development and leadership behavior. 
A number of papers and publications have described these results, for example, Andrew 
(1990), Andrew and Schwab (1993,1995), Baker (1993), and Barton, Andrew and 
Schwab (1994). 

When we used multivariate analysis of the data from all graduates in the 1 1 
institutions in relation to two efficacy items on the Survey of Graduates, Items #19A and 
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#19D, we found some useful predictors : graduates who have positive feelings of 
efficacy tend not to rely on traditional teaching methods such as lecture; instead they 
focus on individualized instruction and cooperative learning. They are leaders among 
their peers. They have strong feelings of making a personal contribution to society, love 
their material, and have positive views of student attitudes. We also found that factors 
influencing the graduates’ sense of efficacy tend to be personally- and classroom-related, 
not school-related (Barton & Oja, 1999). 

How Does UNH Rate Compared to Other Institutions? 

We also used the data from all graduates of the 1 1 institutions as a frame of 
reference to which we could compare the data from UNH graduates and to determine 
how UNH rates compared to all other institutions. For example, Item #34 was one of 
the questions added to the survey that was used with graduates from 1990-1995. We 
found that UNH graduates rate their teacher education program very highly (63.7% of 
UNH graduates rate the UNH program as very good to excellent) compared to graduates 
of all 1 1 institutions (51.8% of graduates of all 1 1 institutions rate their programs as very 
good to excellent). UNH is in the top 2 of the 1 1 institutions for graduate rating of their 
teacher education programs. UNH recently surveyed our graduates from 1995-2000, and 
we find consistency on our graduates’ rating of the UNH teacher education program. 

1900- 1 1995-2000 

UNH Graduates Rating of I JNH T.Ed. Program ( very good to excellent) 63.7% 65.1% 



For another example, we refer to Item #19A that asks about graduates beliefs 
about general teacher efficacy. Item #19A reads: “when it comes right down to it, a 
teacher really can’t do much because most of a student’s motivation and performance 
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depends on his or her own environment.” If a graduate strongly disagrees with item 
#19A, the graduate is described as having a strong belief in general teacher efficacy. We 
found that data from 1985-1990 from UNH graduates differed from the data on All 
graduates of the 1 1 institutions. On this item, 24.4% of UNH graduates from 1985-1990 
strongly believed that teachers, in general, could influence student behavior or 
performance. UNH graduates were similar to graduates from all 11 institutions in the 
1990-1995 sample. When UNH surveyed our graduates from 1995-2000 we again found 
consistency with the UNH data similar to the UNH graduates of the ten prior years. 

#19A General Teacher Efficacy i985-i990 1990-1995 1995-2000 

All graduates of 1 1 institutions 17.2% 29% no data 

UNH graduates 24.4% 27.6% 27.5% 

How Have UNH Graduates Changed Over the Years of the Study? 

We also looked at the data from UNH graduates over the fifteen years that the 
survey has been used, as we wondered about how UNH graduates have changed over the 
years of the study. 

A summary of the UNH Graduates entry and retention data shows consistency 
over 15 years. 





1985-1990 


1990-1995 


1995-2000 


Entry into the orofession (Taught at least one year) 


93 % 


87% 


93% 


Retention ('Current! v teachine/or in a Teaching Related Job) 


90% 


85.3% 


90% 



Examples of changes in other selected variables are shown below. 




8 



BEST COPY AVAILABLE 



8 



Few UNH graduates strongly agree that in their school there is sufficient time 
available for teachers to meet together. 



#1 1 A Time for Teachers to Meet) strongly agree 



1985-1990 1990-1995 1995-2000 

5.7% 12% 5.6% 



Only 34% of UNH graduates from 1985-1990 and 37% of UNH graduates from 
1990-1995 strongly agree that in their school teachers are involved in curriculum decision 
making; and the frequency has decreased to 22% in the 1995-2000 UNH graduate 
sample. 

1985-1990 1990-1995 1995-2000 

- #1 1 F Teachers nre Involved in Curriculum strongly agree 34% 37 % 22% 



In terms of teacher leadership, an expressed goal of the UNH Teacher Education 
Program, when we compared ourselves to the total data from all 1 1 institutions, we did 
not see evidence of how UNH graduates stood out. Frequency data indicates that only 
35% of UNH graduates from the 1990-95 sample to 37% of UNH graduates from 1995- 
2000 data rate the UNH program very good to excellent in preparing them for leadership 
roles to improve schools. 

1985-1990 1990-1995 1995-2000 

#35 Preparation for Leadership Roles (very good to excellent no data 35% 37% 

This self-report data from the UNH graduates is in contrast to the ratings from the school 
principals where the UNH graduates teach. The results of the Teacher Effectiveness 
Survey indicate that the school principals of UNH graduates rate the UNH graduates in 
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the top quartile in terms of instructional performance and leadership. The UNH Teacher 
Education Program now wants to do follow up studies with our UNH graduates in the 
schools using qualitatiive and observational methods to investigate leadership behaviors 
that principals indicated for UNH graduates but that UNH graduates do not self-report. 

Summary 

By collaborating with a consortium of teacher preparation institutions UNH has 
developed a frame of reference for program assessment data. UNH faculty continue to 
analyze results of the Survey of Graduates from 1985-1990, 1990-1995. We have also 
just begun to review the data on UNH graduates from 1995-2000. 
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